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PREFACE 


This  study  was  conducted  for  the  Office  of  the  Under  Secretary  of  Defense  (Per¬ 
sonnel  and  Readiness)  (OUSD(P&R))  under  the  “Advanced  Distributed  Learning 
Common  Framework”  task.  Technical  cognizance  for  this  task  was  assigned  to  Dr.  Robert 
Wisher. 
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EXECUTIVE  SUMMARY 


This  document  briefly  reports  on  an  effort  to  identify  some  directions  that  the 
Advanced  Distributed  Learning  (ADL)  technologies,  particularly  those  designated  as  Intelli¬ 
gent  Tutoring  Systems  (ITSs),  may  take  as  their  capabilities  and  the  capabilities  of  the 
World  Wide  Web  evolve.  Findings  of  this  effort  are  that 

•  Tutorial,  one-on-one  instruction  appears  to  be  substantially  superior  to  class¬ 
room,  one-on-many  instruction.  Interactive  instructional  technology  based  on 
computer  technology  may  make  tutorial  instruction  economically  practical. 

•  Findings  from  many  studies  across  different  subject  matters  suggest  a  “rule  of 
thirds,”  indicating  that  interactive  instructional  technology  can  reduce  the  cost 
of  instruction  by  about  one-third  and  also  either  reduce  the  time  of  instruction 
by  about  one-third  or  increase  learning  by  about  one-third. 

•  ITSs  are  distinguished  from  other  applications  of  interactive  instructional  tech¬ 
nology  by  their  capabilities  to  permit  mixed  initiative  dialogue  in  which  either 
the  technology  or  the  student  can  initiate  questions  and  to  generate  instructional 
material,  strategies,  and  interactions  on  demand  in  real  time. 

-  Both  ITS  capabilities  require  the  availability  of  sharable  instructional 
objects.  Specification  and  development  of  the  objects  are  goals  of  the  ADL 
initiative,  which  is  intended  to  make  high-quality  education,  training,  and 
performance  aiding  available  anytime  and  anywhere.  These  sharable 
instructional  objects,  along  with  their  packaging,  aggregation,  and 
sequencing,  are  being  specified  by  the  ADL  Sharable  Content  Object  Ref¬ 
erence  Model  (SCORM),  which  will  provide  a  foundation  for  affordable, 
adaptive,  and  intelligent  learning  systems. 

•  One  way— perhaps  the  only  way— for  education,  training,  and  performance 
materials  to  be  made  available  anytime  and  anywhere  is  to  take  advantage  of  the 
World  Wide  Web  and  its  services. 

•  Development  of  semantic  capabilities  and  services  will  dramatically  increase  the 
capabilities  of  the  Web,  Web  services,  and  Web  applications,  such  as  ADL,  to 
achieve  their  goals.  Web-based  semantic  capabilities  will  allow  automated 
development  of  more  comprehensive  and  accurate  models  of  subject  matter, 
more  substantive  models  of  learners  and  their  evolving  levels  of  mastery,  and, 
through  enhanced  discovery  and  resolution,  more  precise  identification  of 
instructional  objects  needed  to  produce  desired  levels  of  human  competence. 
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•  Discovery  and  resolution  services  will  be  vastly  improved  through  the  use  of 
Universal  Resource  Names  (URNs),  which  will  serve  as  persistent,  location- 
independent,  resource  identifiers.  These  URNs  will  be  made  possible  by  appli¬ 
cations  of  the  Handle  System  developed  at  the  Corporation  for  National 
Research  Initiatives  (CNRI). 

Developments  such  as  these  will  allow  instructional  programs  to  assemble  continu¬ 
ously  and  unobtrusively  models  of  each  learner’s  state,  style  of  learning,  and  progress 
toward  objectives  and,  in  turn,  will  provide  the  precise,  on-demand  tailoring  of  instructional 
interactions  characteristic  of  one-on-one  tutoring.  Such  a  capability  may  produce  a  third 
revolution  in  anytime,  anywhere  learning  equivalent  to  the  development  of  written  language 
and,  later,  of  books  printed  from  movable  type. 
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OPPORTUNITIES  FOR  NEW  “SMART”  LEARNING 
ENVIRONMENTS  ENABLED  BY 
NEXT-GENERATION  WEB  CAPABILITIES 


A.  INTRODUCTION 

How  might  the  emerging  standards  and  infrastructure  for  Web-based  learning 
objects  enable,  or  even  encourage,  the  development  of  sophisticated  learning  environments 
that  evolve  into  intelligent  tutoring  systems  (ITSs)?  This  question,  which  roughly  identifies 
the  topic  of  this  document,  raises  plenty  of  questions  of  its  own. 

B.  SOME  QUESTIONS 

Let  us  suppose  that  these  learning  objects  do  enable  the  development  of  ITSs.  Are 
technology-based  learning  environments  worthwhile?  What  do  we  mean  by  ITSs  and  What 
do  they  add?  What  emerging  standards,  infrastructure,  and  learning  objects  are  needed  for 
these  systems?  How  might  the  Web  and  Web  services  impact  this  evolution?  Where  might 
these  capabilities  take  us? 

1.  Are  Technology-based  Learning  Environments  Worthwhile? 

As  discussed  in  more  detail  by  Fletcher  (2002),  the  case  for  using  technology  to 
create  these  learning  environments  can  be  roughly  summarized  as  follows: 

•  Tailoring  instruction  (education  and  training)  to  the  needs  of  individual  students 
has  been  found  to  be  an  instructional  imperative  and  an  economic  impossibility. 
Research  has  determined  that  students  tutored  one-on-one  score  about  2  stan¬ 
dard  deviations  higher  on  end-of-course  achievement  tests  than  students  taught 
in  one-on-many  classrooms  (Bloom,  1984).  However,  and  except  for  a  few 
critical  skills  (e.g.,  airplane  piloting,  surgery),  we  cannot  afford  the  one  instruc¬ 
tor  for  each  student  that  such  tutoring  requires. 

•  Technology-based  instruction  can,  in  many  cases,  make  this  instructional 
imperative  affordable.  Under  any  appreciable  student  load,  providing  instruc¬ 
tion  with  technology  is  less  expensive  than  hiring  a  tutor  for  each  student. 

•  Technology-based  instruction  has  been  found  to  be  more  effective  than  current 
classroom  instructional  approaches  in  many  settings  and  across  many  subject 
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matters.  A  review  of  233  evaluations  of  typical  technology-based  instruction 
found  an  average  improvement  of  0.39  standard  deviations  over  classroom 
instruction,  which  is  roughly  equivalent  to  raising  the  performance  of  50th  per¬ 
centile  students  to  that  of  65th  percentile  students.  Review  of  44  similar  evalua¬ 
tions  of  interactive  multimedia  instruction  found  an  average  improvement  of 
0.50  standard  deviations,  roughly  raising  the  performance  of  50th  percentile 
students  to  that  of  69th  percentile  students.  ITSs  have  produced  improvements 
of  1.05  standard  deviations,  roughly  raising  the  performance  of  50th  percentile 
students  to  that  of  about  85th  percentile  students. 

Technology-based  instruction  is  generally  less  costly  than  current  instructional 
approaches,  especially  when  many  students  or  expensive  devices  are  involved. 
In  16  studies  where  achievement  under  technology-based  training  was  at  least 
equal  (and  mostly  superior)  to  that  of  classroom  instruction,  the  cost  ratios  of 
the  former  to  the  latter  were  found  to  be  0.43  for  initial  investment,  0.16  for 
operating  and  support,  and  0.35  overall. 

Technology-based  instruction  has  been  found  to  decrease  the  time  needed  to 
reach  targeted  instructional  objectives.  A  review  of  40  studies  found  that 
savings  in  the  time  needed  to  achieve  given  instructional  objectives  averaged 
about  30  percent.  The  Department  of  Defense  (DoD)  could  save  hundreds  of 
millions  of  dollars  in  specialized  skill  training  if  training  time  could  be  reduced 
by  30  percent.  Time  savings  may  even  more  important  in  K-16  civilian  educa¬ 
tion,  where  opportunities  for  students  to  expand  their  capabilities  and  develop 
their  potential  as  rapidly  as  possible  lay  the  foundation  for  fully  realized,  satis¬ 
fying  lives  as  well  as  global  competitiveness  and  economic  health. 

Technology-based  instruction  has  been  found  to  be  a  cost-effective  alternative 
for  achieving  instructional  goals.  Compared  with  reducing  class  size,  providing 
professional  tutors,  using  peer  tutors,  or  increasing  the  length  of  the  school  day, 
the  costs  to  provide  technology-based  instruction  for  15  minutes  each  day  were 
found  to  be  the  least  expensive  means  for  raising  comprehensive  mathematics 
scores  on  a  standardized  test. 

Technology-based  instruction  will  become  increasingly  affordable  and  instruc- 
tionally  effective  with  the  development  and  use  of  standardized,  reusable 
instructional  objects.  Early  results  have  already  indicated  significant  savings 
(Dodds,  2002). 

The  knowledge  structures  underlying  technology-based  instruction  can  be 
readily  (i.e.,  inexpensively)  used  to  provide  interactive  performance  aiding  that 
both  lowers  training  costs  and  enhances  job  performance  (Fletcher  and 
Johnston,  2002). 


Overall,  a  rule  of  “thirds”  emerges  from  assessments  of  technology-based  instruc¬ 
tion.  Use  of  these  technologies  reduces  the  cost  of  instruction  by  about  one-third,  and  it 
either  reduces  time  of  instruction  by  about  one-third  or  it  increases  the  amount  of  skills  and 
knowledge  acquired  by  about  one-third. 

Technology-based  instruction  can  be  used  either  by  individuals  or  groups  of  indi¬ 
viduals  working  in  collaboration.  It  can  be  used  in  residential  classrooms,  remote  class¬ 
rooms,  or  any  remote  (distributed)  location  (e.g.,  the  workplace,  home,  or  elsewhere).  It  can 
be  available  anytime,  anywhere. 

2.  What  Do  We  Mean  by  ITSs  and  What  Do  They  Add? 

The  features  that  garden-variety  technology-based  instruction  provide  are  notable.  It 
can 

•  Accommodate  individual  students’  rate  of  progress,  allowing  as  much  or  as 
little  time  needed  by  each  student  to  reach  instructional  objectives 

•  Tailor  the  content  and  the  sequence  of  instructional  content  to  each  student’s 
needs 

•  Make  the  instruction  as  easy  or  difficult,  specific  or  abstract,  and  applied  or 
theoretical,  as  necessary 

•  Adjust  to  the  student’s  most  efficient  learning  styles  (collaborative  or  individ¬ 
ual,  verbal  or  visual,  and  so  forth).  These  capabilities  have  been  available  and 
used  in  technology-based  instruction  from  its  inception  in  the  1950s  (Fletcher 
and  Rockway,  1986). 

The  term  “intelligent”  in  an  ITS  refers  as  much  to  intentions  as  to  results,  but  it  is 
more  than  a  marketing  term.  It  refers  to  specific  capabilities  that  have  been  the  goals  of  ITS 
development  since  these  systems  were  first  attempted  in  the  1960s  (Carbonell,  1970;  Brown, 
Burton,  and  DeKleer,  1982).  Two  defining  capabilities  of  ITSs  are  that  they 

1.  Allow  either  the  system  or  the  student  to  ask  open-ended  questions  and  initiate 
instructional,  “mixed-initiative”  dialogue  as  needed  or  desired 

2.  Generate  instructional  material  and  interactions  on  demand  rather  than  requiring 
developers  to  foresee  and  pre- store  all  such  materials  and  interactions  needed  to 
meet  all  possible  eventualities. 

Mixed-initiative  dialogue  requires  a  language  that  is  shared  by  the  system  and  by  the 
student/user  for  information  retrieval,  decision-aiding,  and  instruction.  Natural  language 
(NL)  has  been  a  frequent  choice  for  this  capability  (e.g..  Brown,  Burton,  and  DeKleer,  1982; 
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Graesser,  Person,  and  Magliano,  1995),  but  the  language  of  mathematics,  mathematical 
logic,  electronics,  and  other  well- structured  communication  systems  has  also  been  used 
(Suppes,  1981;  Sleeman  and  Brown,  1982;  Psotka,  Massey,  and  Mutter,  1988). 

The  generative  capability  requires  the  system  to  devise  the  interactions  with  students 
on  demand— not  draw  from  predicted  and  pre-stored  formats.  This  capability  involves  not 
only  generating  problems  tailored  to  each  student’s  needs,  but  also  providing  coaching, 
hints,  critiques  of  completed  solutions,  appropriate  and  effective  teaching  strategies,  and, 
overall,  the  interactions  and  presentations  needed  for  one-on-one  tutorial  instruction.  These 
interactions  must  be  generated  from  information  primitives  using  an  “instructional  gram¬ 
mar”  that  is  analogous  to  the  deep- structure  grammar  of  linguistics. 

Motivations  for  these  two  capabilities  can  be  found  in  basic  research  on  human 
learning,  memory,  perception,  and  cognition.  Findings  from  this  research  have  led  us  to  view 
all  cognitive  processes  as  constructive  and  regenerative  (Neisser,  1967).  These  findings  have 
also  extended  general  theories  of  perception  and  learning  from  the  fairly  strict  logical  posi¬ 
tivism  of  behavioral  psychology,  which  emphasized  directly  observable  actions,  to  a  consid¬ 
eration  of  the  internal,  cognitive  processes  that  are  assumed  to  mediate  and  enable  human 
learning.  The  hallmark  of  these  conceptions  of  cognition  is  that  seeing,  hearing,  and  remem¬ 
bering  are  all  acts  of  construction,  making  more  or  less  use  of  the  limited  stimulus  informa¬ 
tion  provided  by  human  perceptual  capabilities. 

The  generative  capability  sought  by  ITS  developers  is  not  something  merely  nice  to 
have,  but  it  is  essential  if  we  are  to  advance  beyond  the  constraints  of  prescribed,  pre¬ 
branched,  programmed  learning  and  the  ad-hoc  principles  commonly  used  to  design  tech¬ 
nology-based  instruction.  We  need  this  capability  if  we  are  to  deal  successfully  with  the 
immensity,  extent,  and  variability  of  human  cognition. 

3.  What  Emerging  Standards,  Infrastructure,  and  Learning  Objects  Are  Needed 
For  These  Systems? 

Specification  for  learning  objects  has  become  an  essential  component  of  the 
Advanced  Distributed  Learning  (ADL)  initiative.  This  initiative  is  the  most  recent  and  visible 
effort  in  a  long  campaign  to  adopt  the  benefits  of  technology-based  instruction  and  per¬ 
formance  aiding  in  routine  practice.  It  is  intended  to  accelerate  large-scale  development  of 
dynamic  and  cost-effective  learning  software  and  to  stimulate  a  vigorous  market  for  learning 
software.  Its  goal  is  to  ensure  access  to  high-quality  “learning”  (education,  training,  and 
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performance  aiding)  that  is  tailored  to  individual  needs  and  capabilities  and  available  at  any¬ 
time  and  anyplace  (Dodds,  2002). 

The  ADL  initiative  is  preparing  for  a  future  in  which  communication  networks  and 
personal  delivery  devices  are  pervasive,  inexpensive,  and  effectively  transparent  to  users 
through  ease  of  use,  expanded  bandwidth,  and  portability.  It  will  establish  knowledge  librar¬ 
ies,  or  repositories,  where  learning  “objects”  can  be  accumulated  and  cataloged  for  broad 
distribution  and  use.  These  objects,  because  of  their  enhanced  accessibility,  will  be  ready  for 
assembly  on  demand  and  in  real  time  into  instructional  and  performance-aiding  materials 
that  are  tailored  to  the  capabilities,  intentions,  and  learning  state  of  each  individual  or  group 
of  individuals  needing  them  (Dodds,  2002). 

ADL  and  ITSs,  therefore,  have  several  key  goals  in  common: 

•  Both  are  generative  in  that  they  seek  to  prepare  and  present  interactions  on 
demand,  in  real  time. 

•  Both  are  intended  to  provide  instructional  interactions  that  are  tailored  in  con¬ 
tent,  sequence,  difficulty,  style,  and  so  forth  to  users’  intentions,  backgrounds, 
and  needs. 

•  Both  have  a  stake  in  research  intended  to  accomplish  such  individualization. 

•  Both  can  be  used  equally  well  in  instruction  and  performance  aiding. 

•  Both  are  intended  to  accommodate  mixed-initiative  dialogue  in  which  either  the 
technology  or  the  user  initiates  or  responds  to  open-ended  inquiry  and 
discussion. 

•  Both  require  a  supply  of  sharable  instructional  objects  that  are  readily  available 
for  the  generation  of  instruction  or  performance-aiding  presentations. 

To  date,  most  of  the  ADL  effort  has  been  devoted  to  the  specification  of  instruc¬ 
tional  objects  that  will  populate  learning  libraries  and  other  Web-available  repositories. 
These  objects  are  separated  from  context-specific  run-time  constraints  and  proprietary  sys¬ 
tems  so  that  they  can  be  incorporated  into  other  applications.  They  have  common  interfaces 
and  data  exchange  formats.  They  are  accessible  so  that  they  can  be  indexed  and  readily 
found  or  “discovered”;  interoperable  so  that  they  operate  across  a  wide  variety  of  hard¬ 
ware,  operating  systems,  and  Web  browsers;  durable  so  that  they  do  not  require  modifica¬ 
tion  as  versions  of  the  underlying  software  systems  change;  and  reusable  so  that  they  can 
be  adapted  and  used  by  many  different  development  tools.  The  ADL  initiative  has  coordi¬ 
nated  groups  of  industry,  academic,  and  government  stakeholders  working  together  to 
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specify  objects  that  meet  these  criteria.  These  specifications  have  produced  evolving,  cumu¬ 
lative  versions  of  the  Sharable  Content  Object  Reference  Model  (SCORM)  (Dodds,  2002). 

As  presently  defined,  SCORM  objects  can  be  entire  courses,  lessons  within  courses, 
or  modules  within  lessons.  Their  granularity  or  size  remains  an  issue.  ADL  development  is 
presently  focused  on  packaging  these  objects  for  what  has  been  called  the  “educational 
object  economy”  (Spohrer,  Sumner,  and  Shum,  1998).  One  idea  behind  such  an  economy 
is  that  the  emphasis  in  preparing  materials  for  technology-based  instruction  (or  decision- 
aiding)  will  shift  from  the  current  concern  with  preparing  content  components,  or  instruc¬ 
tional  objects,  to  one  of  integrating  already  available  content  into  meaningful  and  relevant 
presentations. 

Technicians,  software  engineers,  instruction  designers,  and  cognitive  researchers 
from  all  sectors  of  the  economy  in  the  Americas,  Europe,  and  Asia  have  joined  in  this  quest. 
The  task  of  specifying  and  developing  these  objects  has  become  a  global  effort.  The  pri¬ 
mary  contribution  of  ADL  has  been  to  orchestrate  this  effort  and  document  the  results. 

Toward  More  Adaptive,  Intelligent  Learning  Systems 

Until  recently,  most  mainstream  learning  systems  have  relied  on  predetermined  and 
often  fixed-path  delivery  of  content.  Such  systems  lack  agility  in  adapting  to  learners’  mas¬ 
tery  states  and  are  thereby  limited  in  their  ability  to  tailor  learning  experiences  to  individual 
learners.  An  adaptive,  “intelligent”  learning  system  needs  an  accurate  model  of  the  learner, 
a  model  of  the  knowledge  domain,  and  a  capability  that  can  evaluate  the  differences  between 
the  two  models  and  identify  or  devise  (on-demand  and  in  real  time)  instructional  strategies 
that  will  achieve  desired  instructional  outcomes. 

SCORM  presently  provides  a  rules-based  “learning  strategy”  that  enables  sharable 
content  objects  (SCOs)  to  set  the  state  of  globally  accessible  records.  These  records  can 
store  the  learner’s  degree  of  mastery  in  the  form  of  a  score  or  a  pass/fail  state  or  can  store 
the  progress  of  the  learner  in  terms  of  completion.  A  “hook”  was  included  in  the  records 
that  permits  them  to  reference  externally  defined  competencies.  As  the  learner  is  sequenced 
through  the  SCOs,  the  learning  system  constructs  a  representation  of  the  learner’s  mastery 
and  progress.  The  objective  records  can  be  viewed  as  a  simple  model  of  the  learner’s  state 
(Gibbons  and  Fairweather,  1998).  How  far  can  SCORM’ s  sequencing,  navigation,  and 
assessment  capabilities  can  be  pushed  in  the  direction  of  generative  intelligent  learning  sys¬ 
tems?  This  remains  an  empirical  issue  that  is  currently  being  settled  in  applications  of 
increasingly  sophisticated  learning  systems  observing  SCORM  specifications. 
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Another  emerging  specification  called  the  IMS  Reusable  Definition  of  Competency 
or  Educational  Objective  (IMS  Global  Learning  Consortium,  Inc.,  2002)  defines  a  means 
of  building  a  taxonomy  of  competency  definitions  that  meet  specific  objectives.  This  taxon¬ 
omy  can  be  organized  hierarchically  to  represent  dependencies,  supporting  skills,  or  prereq¬ 
uisites.  Each  competency  definition  has  a  text  description  of  the  competency  and  a  unique 
identifier  that  can  be  referenced  externally.  The  organization  of  a  competency  definition 
could  represent  specific  skills  or  knowledge  to  be  acquired  for  a  specific  task  or  subject 
domain  (e.g.,  as  one  might  find  in  Quantitative  Domain  Mapping).  Since  objectives  records 
in  SCORM  can  reference  the  competency  model  identifiers,  the  means  to  compare  the  state 
of  the  learner  and  the  desired  competencies  now  exist.  This  capability  provides  a  system- 
based  means  to  perform  knowledge  and  skills  gap  analyses  that  will  lead  to  more  sophisti¬ 
cated  and  adaptive  strategies  that  use  such  information  (Wiley,  2000). 

As  learning  system  specifications  become  more  robust,  they  will  also  become  more 
adaptive.  Improved  assessment  methods  and  results  that  will  continuously  and  unobtru¬ 
sively  extract  information  from  instructional  interactions  and  better  represent  the  state  of  the 
learner  are  emerging  (e.g.,  Conati,  et  al.  1997;  Corbett,  Koedinger,  and  Anderson,  1997). 
The  strategies  developed  by  learning  systems  will  be  informed  further  by  learner  profile 
information,  which  can  “pre-load”  the  learner  model  with  mastery  information  from  out¬ 
side  of  the  system.  This  process  will  improve  the  processes  used  by  technology-based 
instructional  systems  to  bypass  relevant  content  of  pre-mastered  material  (e.g.,  holders  of 
certificates  in  particular  subjects)  and  concentrate  on  relevant  material  yet  to  be  learned. 

The  emerging  specifications  have  provided  a  means  of  modeling  and  tracking  the 
learner  and  referencing  external  models  of  knowledge  and  competency.  The  specifications 
now  allow  conditional  rules  that  can  tailor  what  the  learner  experiences  to  his/her  mastery 
and  progress  (e.g.,  Dodds,  2002).  Future  services  and  processes  will  extend  these  basic 
capabilities  in  more  sophisticated  and  nuanced  ways. 

Basically,  what  we  seek  is  an  engineering  of  instruction  in  which  all  learners  reliably 
achieve  outcomes  such  as  the  retention  of  skills  and  knowledge,  the  transfer  of  learning  to 
new  but  similar  applications,  the  motivation  to  continue  study,  the  speed  and  accuracy  of 
response,  and  so  forth.  Such  an  engineering  of  instruction  would  adjust  and  modulate  the 
learning  experience  for  each  individual.  It  would  identify  the  learner’s  characteristics,  level 
of  knowledge,  and  style  of  learning;  devise  learning  strategies  as  described  earlier  and  later 
provide  these  learning  strategies  on-demand  and  in  real  time  for  each  successive  interaction 
with  the  learner;  locate  objects  that  are  appropriate  to  the  outcome  being  sought;  and  specify 
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the  instructional  strategy  that  was  identified  or  devised.  This  is  a  significant  challenge  for 
the  developers  of  instructional  objects  and  for  Web-based  services,  but  current  work  sug¬ 
gests  that  they  can  meet  it  successfully. 

4.  How  Might  the  Web  and  Web  Services  Impact  This  Evolution? 

One  way  the  current  and  near-term  capabilities  of  learning  systems  might  evolve  is 
through  the  Semantic  Web,  which  will  provide  powerful  new  technologies  for  both  knowl¬ 
edge  representation  and  the  ontologies  needed  to  connect  them  (Heflin,  2003).  These  tech¬ 
nologies  will  provide  ways  not  only  to  relate,  but  also  to  reason  about  information  from 
widely  different  domains. 

The  Semantic  Web  is  intended  to  imbue  information  available  on  the  Web  with  suf¬ 
ficient  meaning  to  improve  substantially  the  cooperation  between  computers  and  human 
beings.  It  requires  abstract  representation  of  information  on  the  Web  using  a  Resource 
Description  Framework  and  other  specifications  that  have  not  yet  been  developed.  Dealing 
with  the  semantic  content  of  Web  pages  and  information  will  improve  the  process  of  dis¬ 
covery  needed  to  access  relevant  information  and  objects  from  the  Web.  Through  an  ontol¬ 
ogy  that  consists  of  a  taxonomy  and  a  set  of  inference  rules  that  formally  define  operations 
and  relations  among  terms,  identifying  and  exposing  semantic  linkages  between  highly  dis¬ 
parate  bodies  of  information  will  be  possible. 

If  the  Semantic  Web  is  successful,  it  will  integrate  real-world  knowledge  and  skills 
acquired  through  simulation,  education,  training,  performance  aiding,  and  experience.  It  will 
provide  a  foundation  for  building  more  comprehensive  and  substantive  models  of  subject 
matter  domains  and  learners’  levels  of  mastery  than  we  now  have  and  will  combine  them 
with  more  precise  discovery  of  the  instructional  objects  needed  to  produce  desired  human 
competencies.  Learners  and  practitioners  will  be  presented  with  a  constellation  of  related 
activities  (e.g.,  learning,  doing,  trying,  referencing).  This  integration,  combined  with  the 
already  available  functionalities  of  ITSs,  provides  the  basis  for  a  next-generation  meta-archi- 
tecture  and  learning  environments  based  on  instructional  objects. 

Core  components  of  the  Semantic  Web  will  be  built  on  top  of  existing  and  emerging 
Web  standards.  These  standards  provide  the  means  to  express  the  complex  relationships 
and  inference  rules  processed  by  Web  services  to  perform  specific  tasks,  such  as  profiling 
learners;  representing  their  skills,  knowledge,  and  abilities;  linking  these  representations  to 
instructional  objects;  and  managing  their  progress  toward  objectives  and  competencies. 
Web  services  will  serve  as  reusable,  black-box  applications  that  generate  other  Web-based 
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applications  from  objects.  They  will  use  open  Internet  standards,  such  as  Hyper-Text  Trans¬ 
fer  Protocol  (HTTP),  extensible  Markup  Language  (XML),  Universal  Description  Discov¬ 
ery,  and  Integration  (UDDI),  and  Simple  Object  Access  Protocol  (SOAP)  to  exchange 
information  between  applications  as  needed.  The  services  will  be  language,  platform,  and 
object  model  independent.  They  will  enable  different  applications,  which  run  on  different 
operating  systems  and  are  developed  with  different  object  models  using  different  program¬ 
ming  languages,  to  cooperate  and  become  easily  used  Web  applications.  Taking  advantage 
of  existing  infrastructure  and  applications,  they  will  provide  flexible,  standards-based  capa¬ 
bilities  for  binding  applications  together  over  the  Internet. 

Content  Object  Discovery  and  Retrieval 

Searching  and  discovering  contextually  relevant  instructional  content  have  become 
major  topics.  The  success  of  Google  and  other  Web  search  engines  have  whetted  every¬ 
one’s  appetite  for  “just  when  you  want  it”  search  and  retrieval  and  has  demonstrated  the 
value  and  utility  of  content  discovery.  Presently,  Google  may  be  the  most  important,  effec¬ 
tive,  and  widely  used  source  of  Web-based  education.  However,  Google’s  method  of 
locating  content  only  by  text  crawling  and  indexing  and  by  retrieving  anything  that  is  avail¬ 
able  and  remotely  relevant  has  limited  its  use  as  a  discovery  system  for  just-in-time,  focused 
content  assembly.  Its  operation  could  be  improved  substantially  if  it  were  to  “cooperate” 
with  content  and  retrieve  only  what  is  intentionally  prepared  and  published  for  discovery. 

A  series  of  Internet  specifications,  which  have  been  in  development  for  some  time, 
appear  to  be  maturing  and  might  form  a  framework  on  which  sophisticated  search,  authenti¬ 
cation,  accreditation,  and  resolution  services  might  be  built.  These  services  will  provide  a 
comprehensive  means  for  instructional  programs  to  locate  and  access  appropriate  content. 
At  least  two  capabilities  are  needed  to  build  such  services: 

1.  Content  object  identification  and  resolution.  Content  object  identification 
(to  select  candidate  objects)  and  resolution  (to  narrow  the  selection  to  objects 
that  are  precisely  relevant)  are  being  addressed  through  the  use  of  Universal 
Resource  Names  (URNs),  which  are  intended  to  serve  as  persistent,  location- 
independent,  resource  identifiers.  The  Corporation  for  National  Research  Ini¬ 
tiatives  (CNRI)  has  created  a  URN  implementation  called  “The  Handle  Sys¬ 
tem”  (Kahn  and  Wilensky,  1995;  CNRI,  2003),  which  allows  digital  objects  to 
obtain  a  unique  identifier  and  link  each  object  to  its  location— wherever  that 
might  be— through  the  use  of  a  Handle  Resolution  Service  (similar  to  domain 
names  resolving  to  Internet  protocol  addresses  through  the  Domain  Name 
System).  CNRI  hosts  a  global  root  server  that  can  be  queried  during  resolution 
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requests.  The  Handle  System  addresses  a  key  repository  problem:  the  unique 
identification  of  objects  along  with  their  present  location  and  descriptive 
metadata. 

2.  Discovery  indexing  with  search  criteria.  Discovery  indexing  presents 
another  challenge.  How  does  a  user  locate  relevant  content  in  the  first  place? 
One  approach  may  be  to  use  the  Handle  System  to  set  up  a  registry  and  index 
of  content  repositories  so  that  a  repository  can  be  located  and  searched.  To  do 
so,  some  form  of  external  search  must  be  enabled  against  some  defined  search 
criteria.  The  Global  Information  Locator  Service  (GILS)  specification,  which  is 
based  on  an  International  Standards  Organization  (ISO)  search  standard  (ISO 
23950),1  addresses  how  repositories  might  identify  (discover)  content  within  a 
repository  collection. 

The  Common  Indexing  Protocol  (CIP),  developed  by  the  Internet  Engineering  Task 
Force  (IETF)  (http://www.ietf.org/),  allows  the  owner  of  content  to  create  its  index  metadata 
while  also  allowing  this  indexing  information  to  be  shared  among  different  servers,  thereby 
enabling  the  development  of  new  search  and  discovery  services.  New  learning  and  perform¬ 
ance  aiding  specifications  are  emerging.  These  specifications  permit  the  identification  of 
skills,  competencies,  and  knowledge  so  that  logical  relations  among  them  can  be  made  and 
then  represented  in  taxonomies  that  are  relevant  to  specific  but  quite  different  communities 
of  practices.  One  example  of  these  specifications  is  the  Reusable  Definition  of  Competency 
or  Educational  Objective  specification  from  the  IMS  Global  Learning  Consortium,  Inc., 
(http://www.imsglobal.org/),  which  is  now  being  advanced  to  the  Institute  of  Electrical  and 
Electronic  Engineers  (IEEE)  as  a  candidate  standard. 

These  specifications  show  great  promise  for  the  design  and  deployment  of  Internet 
and  Web  services  that  will  enable  accurate,  precisely  focused,  and  contextually  correct  dis¬ 
covery  and  retrieval  of  learning  content  objects  on  a  highly  scalable  basis  — an  ability  still 
not  yet  available.  They  will  allow  instructional  programs  to  assemble  continuously  and 
unobtrusively  the  models  of  each  learner’s  state  of  knowledge,  style  of  learning,  and  pro¬ 
gress  toward  instructional  objectives.  In  turn,  these  models  will  provide  each  student  the 


1  ISO  23950:  Information  retrieval  (Z39. 50)— Application  Service  Definition  and  Protocol  Specification. 
It  is  the  international  standard  for  information  search,  equivalent  to  the  U.S.  standard  designated  Ameri¬ 
can  National  Standards  Institute/National  Information  Standards  Organization  (ANSI/NISO)  Z39.50. 
ISO  23950  was  developed  primarily  in  the  library  and  information  services  communities,  though  now 
it  is  broadly  applied  worldwide  on  the  Internet  and  other  networks.  It  supports  full-text  search  but  also 
supports  large,  complex  information  collections.  While  ISO  23950  leverages  common  practice,  it  does 
not  enforce  any  particular  format.  It  does  not  specify  how  network  servers  manage  records  or  how 
clients  use  records.  It  only  specifies  how  to  express  a  search  and  return  results. 
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precise  tailoring  of  instructional  interactions  that  is  a  characteristic  and  unique  strength  of 
one-on-one  tutoring.  The  next  several  years  are  likely  to  see  a  great  deal  of  emphasis  on 
developing  the  specifications  and  services  needed  to  make  this  possible. 

5.  Where  Might  These  Capabilities  Take  Us? 

The  emphasis  on  instructional  technology  causes  us  to  reflect  upon  revolutions  in 
instruction.  The  first  revolutions  in  instruction  may  have  occurred  with  the  development  of 
written  language  about  7,000  years  ago.  This  allowed  the  content  of  advanced  ideas  and 
teaching  to  transcend  time  and  place.  The  second  revolution  in  instruction  began  with  the 
technology  of  books.  Books  made  the  content  of  high-quality  instruction  available  any¬ 
where  and  anytime  in  an  inexpensive  medium  that  was  accessible  to  many  more  people.  A 
third  revolution  in  instruction  appears  to  be  accompanying  the  introduction  of  computer 
technology.  The  capability  of  this  technology  for  real-time  adjustment  of  instructional  con¬ 
tent,  sequence,  scope,  difficulty,  and  style  to  meet  the  needs  of  individuals  suggests  a  third 
pervasive  and  significant  revolution  in  instruction.  It  makes  both  the  content  and  the  interac¬ 
tions  of  high-quality  instruction  widely  and  inexpensively  accessible— again,  anytime  and 
anywhere. 

Building  on  this  possibility,  ADL,  SCORM,  intelligent  tutoring,  and  the  Semantic 
Web  will  provide  a  foundation  for  generative  education,  training,  and  performance  aiding 
available  anytime  and  anywhere.  These  developments  will  capitalize  on  the  growth  of  elec¬ 
tronic  commerce  and  the  global  information  grid.  They  will  build  on  this  worldwide,  almost 
irresistible  activity,  accelerate  it,  and  apply  it  to  a  full  spectrum  of  education,  training,  and 
performance  aiding  needs. 
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